Data taken from remote sensors, such as those in spacecraft, is typically received and processed at computer systems located elsewhere. Data such as telemetry data, temperature measurements, or electrical circuit measurements (voltage or current) can be reported as a series of time-ordered points, which is then archived as the data arrives, in time order. Thousands of unique data identifiers may be collected and processed per second, but an operator or analyst may only be interested in a small subset of the archived data. Some existing systems store the data received sequentially on disk, with records ordered by increasing time and without separating them by data type. If the data cannot be stored in a fixed-size record, it is often split into two pieces: an index containing fixed-size data, and a pointer to variable length data.
Retrieving data stored in this format traditionally involves scanning through every record and discarding those that do not meet the requested criteria. This process can waste CPU cycles, disk I/O bandwidth, and time.